In computing, specifically data transmission and data storage, a block is a sequence of bytes or bits, having a nominal length which is also referred to as a block size. The process of putting data into blocks is used to facilitate the handling of the data-stream by the computer program receiving the data. Blocked data are normally read a whole block at a time. This means that there is no means to address the data put inside a block.
Blocks in a block device are typically only 512 bytes large. An operating system that uses this block device would typically write or read 8 blocks at a time. Such a group of 8 blocks is typically referred to as a cluster. As such a cluster is a sequence of blocks that is written and read consecutively.
Block special files or block devices correspond to devices through which the operating system moves data in the form of blocks. Device nodes corresponding to these devices often represent addressable devices such as hard disks, CD-ROM drives, or memory-regions.
Logical Block Addressing (LBA) is a particularly simple linear addressing scheme. Blocks are located by an index, with the first block being LBA=0, the second LBA=1, and so on. The LBA scheme replaces earlier schemes which exposed the physical details of the storage device to the software of the operating system. Chief among these was the cylinder-head-sector (CHS) scheme, where blocks were addressed by means of a tuple which defined the cylinder, head, and sector at which they appeared on the hard disk. Current LBA schemes allow disks of size 128 PetaByte to be addressed, assuming a block size of 512 bytes.
A storage snapshot is a set of reference markers, or pointers, to data stored on a disk drive, on a tape, or in a storage area network (SAN). A snapshot is something like a detailed table of contents, but it is treated by the computer as a complete data backup. Snapshots streamline access to stored data and can speed up the process of data recovery. In current storage technologies, there are two main types of storage snapshot, called the copy-on-write (or low-capacity) snapshot and the split-mirror snapshot. Utilities are available that can automatically generate either type.
A copy-on-write snapshot utility creates a copy of the existing data blocks at another location to store new data blocks in a given location every time existing data blocks are updated. This allows rapid recovery of data in case of a disk write error, corrupted file, or program malfunction. However, all previous snapshots must be available when complete archiving or recovery of all the data on a network or storage medium is needed. The copy operation for every block of data that is updated slows down the write process significantly.
A split-mirror snapshot utility references all the data on a set of mirrored drives. Every time the utility is run, a snapshot is created of the entire volume, not only of the new or updated data. This makes it possible to access data offline, and simplifies the process of recovering, duplicating, or archiving all the data on a drive. However, this is a slower process, and it requires more storage space for each snapshot.
Cloning is the process of providing a copy of a point in time of data storage and allowing write operations on it without modifying the original data storage where this copy has been taken from.
MetaData in a block device and storage context is data that describes data operations: when specific block operations have been executed on what data, in what sequence and where these blocks or block changes are stored and how these can be addressed.
Cloud storage solutions are typically best suited to store large objects, with a size of 1 MB and more (e.g. mp3 files, mpeg files, jpg files, etc.).
In order to make cloud storage available to systems, such as operating systems, file systems, applications, hypervisors, . . . which were developed mainly to interface with block devices there is a need for a device driver that can manage the transfer between a block or clusters and cloud storage container objects in an efficient way.
As such it is an object of the current invention to provide intelligent caching technology such that a sequence of block or cluster writes is grouped and stored in a cloud storage container object with a size that is well suited for a cloud storage system (e.g. 4 MB).